Stabilizing the spiral order with spin-orbit coupling in an anisotropic triangular 

ant iferromagnet 
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We study the effects of spin-orbit coupling (SOC) on the large-U Hubbard model on anisotropic 
triangular lattice at half-filling using the Schwinger-boson method. We find that the SOC will in 
general lead to a zero temperature condensation of the Schwinger bosons with a single condensation 
momentum. As a consequence, the spin-spin correlation vanishes along the z-axis but develops in 
the x-y plane, with the ordering vector being dramatically dependent on the SOC. Moreover, the 
phase boundary of the magnetic ordered state extends to the region of large spatial anisotropy with 
increasing condensation density, demonstrating that the spiral order is always stabilized by the SOC. 
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Antiferromagnets on the triangular lattice represent 
a prototype correlated systems where certain novel 
mangetic phases such as the spin liquid state may 
emerge due to the quantum fluctuations and geometric 
frustrations 1 . Some materials candidates of the spin liq- 
uid, such as the organic k- (BEDT -TTF) 2 Cu 2 (CN) 3 2 
and EtMe3Sb[Pd{dmii)2\2 i , are the triangular-lattice 
antiferromagnets, where the interchain coupling J' is 
close to the intrachain coupling J. Another layered quan- 
tum magnet, CsiCuCl/^, has a relatively large spatial 
anisotropy a = J' /J ~ 0.34. The inelastic neutron scat- 
tering measurements reveal the possible spin liquid phase 
for temperature above Tn — 0.62A", while for T < TV, 
the spiral order is set up£. The emergent spiral order 
is attributed to the interlayer coupling and the small 
Dzyaloshinskii-Moriya (DM) interaction^— which is put 
by hand in the Heisenberg model. On the other hand, 
the SOC is ubiquitous in materials whose crystal struc- 
tures lack the inversion symmetry, and CsiCuCl^ just 
falls into this category. 

Theoretically, though the SOC has been extensively 
studied for the electronic systems with relatively small 
Coulomb interaction U, it remains challenging to under- 
stand the effect of SOC when U is moderate or strong. In 
this paper, we study the Hubbard model on anisotropic 
triangular lattice with the finite SOC and the infinite U. 
The spatial anisotropy a in the studied model is an im- 
portant tuning parameter which interpolates the decou- 
pled chains (a — > 0) and the square lattice (a — > oo). In 
the absence of the SOC, a quasi one-dimensional spin liq- 
uid phase and a two-dimensional magnetic phase emerge 
at the half-filling in the two limiting cases, respectively. 
Thus by increasing a, it is natural to expect a critical a c , 
separating the spin liquid and magnetic ordered states. 
Various numerical calculations for finite clusters^ - — ob- 
tain a c ~ 0.7 — 0.9, while the linear spin-wave theory^ 
predicts a much smaller value a c — 0.27. Here we shall 
mainly focus on the magnetic ordered phase and study 
the influence of the SOC by using the Schwinger-boson 
mean-field theory. 
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FIG. 1: The triangular lattice. (a)The bond vectors are ai,a2 
and a3. The electrons with up spin feel the staggered flux as 
shown in the figure and the electrons with down spin feel the 
flux with opposite direction. (b)The spin coupling strength 
and the phase factor at each bond. 



Our starting point Hamiltonian is the Hubbard model 
on triangular lattice with the SOC, defined by 



H 



<ij>,cr 



[(iij - iagis)cl a c ia + h.c] + UY] n-^n^. (1) 



Where, is the hopping energy between the nearest 
neighboring bond (ij), c\ a annihilates an electron at i 
with the spin index a = +/— for up/down spins, 
represents the strength of the SOC. We shall at first 
drive an equivalent spin model with finite SOC for large 
U and at half-filling. This has been recently realized 
for the same model defined on the kagome lattice^. 
Here the similar approach is illustrated by the two-site 
problem. Noticed that the bond (ij) is oriented, the 
SOC brings an stagger flux through each triangle pla- 
quette as shown in FigOJa). Thus the first term can 
be re-expressed by — £12 J2a( e ~ i<7Dl2c ia c ^ + h.c), with 
*12 = \A?2 +5i2 and D i2 = arctan (312/^12). By per- 
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forming the site and spin-dependent U(l) transforma- 
tion: c Xa = e l,y(Dl2l2) c lrj , c 2(T = e- i(T< - Dl2 /^c 2a , the cor- 
responding Hamiltonian (at the large U limit) is mapped 

to the spin Hamiltonian Ji 2 Si • S2 with Ji 2 = -£p and 

Noticed that the triangular lattice is not a bipartite 
lattice, there exists no compatible £7(1) transformation 
for every site. For a lattice model, we should transform 
back to the original fermion operators. Therefore, the 
spin model for the strong-coupled Hubbard model on the 
half-filled triangular lattice turns out to be 



H 



<ij> 



-2iD 



2iD u a- 



»SrS7) + SfSi 



where 5* = S x ±iS y . The SU(2) symmetry of the above 
spin model is broken by the SOC. And its antisymmetric 
(the DM-term ) and symmetric parts appear with explicit 
SOC-dependent coupling strengths. 

We then study the Hamiltonian © using the 
Schwinger-boson mean-field theory^. The spin oper- 
ators are represented by boson operators S + 



b\b 



S- = b\bf, S z = \{b\b^ - b\b±), with the con- 
straint b^b^ + b^bi — 1. We introduce the mean-fields 
ipna = -iv(biabi +anS ), with ai = a (-5,^), a 2 = 

Oo(— |) - 2 )' a3 = ~( ai + &2 ) = <2o ( 1 ' °)> and fl ° is tllC 

lattice constant, as marked in FigQJa). The obtained 
mean-field Hamiltonian is 



h = ^Y. dJ ^™ + e2lsDn ^^ b \+ anS + h - c - 

\rwj 
i a 

where the A-term is introduced to impose the constraint. 
In the momentum space, the Hamiltonian has the form 



A A k 



27VA, (4) 



where N = Ni X JV-j is the site number of the lattice 
( a! and a 2 are chosen as the two primitive translation 
vectors), A k = iE^^^+e 21 ^^)^* 8 ". We 
assume that is a real number, because the phase factor 
of Ak can be gauged away by the U(l) transformation. 
Due to the time-reversal symmetry, we also have \ipnt\ = 
\ip n i I . Thus we use the ansatz if) nu = ip n e l,T ® n , and obtain 
4 = 2E n Jn sin (k • a„ - D n ) cos (D„ + & n )i>n- 

The mean-field Hamiltonian Eq.(4) is diagonalized by 
the bosonic Bogliulov transformation, b^ — w k a k ^ + 
UkO^, b-kj. = M k a ki + w k a kT , with u\ = ^ + \, 
v l=2^Z-h and w k"k 



2uj 



Then 



H = ^2 Wka^afo, + £(w k - 2A )' 



(5) 



where the quasiparticle dispersion cj k = yj\ 2 — A^. A 
stable ground state requires A > |A k |. The free energy is 
given by 



i7 = ^E ln ( 1 - e ^ k ) + EK-2A) 



(6) 



with j3 = v^f- The Lagrangian multiplier A is deter- 
mined by optimizing the free energy, leading to 



^E^ (*(<■*) + 5) = 1. 

k 



(7) 



i(2) where nf,(w k ) 



— - is the Bose distribution func- 

2 p^k — 1 

tion. The mean-fields t/) n and are then calculated 
self-consistently through following equations, 



^E^(^K) + ^) e 



— ik-j 



(8) 



Keeping in mind that J n = {t 2 n + g„)/U and £>„ = 
arctan (g n /t n ), we set Ji = J 2 = J', J3 = J, and Di = 
D2 = D3 = D as illustrated in FigOJb). Thus the spatial 
anisotropy is measured by a = J' / J. Three special cases 
in the parameter space are (1) a = 0, the decoupled 
chain's limit; (2) a — 00, the square lattice limit; and (3) 
a = 1, the isotropic point. Notice that with this choice of 
parameters, the mean-field equations are invariant under 
the exchange of the bond index 1 and 2. Therefore, we 
have ipi = ip2 and -&i = z? 2 . 

According to the analysis with functional inte- 
gral method 16 , no bosons can condense in the two- 
dimensional lattices at finite temperature. This is consis- 
tent with the Mermin- Wagner theoremil. In the case of 
a gapped energy spectrum, i.e. minw(k) =/= 0, there is no 
Bose-Einstein condensation and the ground state is a spin 
liquid. Otherwise, if the spectrum is gapless, the bosons 
can condense on the lowest energy state, implying a long- 
range magnetic orderJ^ The relation between magnetic 
order and Bose-Einstein condensation can be drawn from 
the spin-spin correlation whose diagonal components are 
given by 



(SZSf) = (SgSf ) 
(«W> = k|/i| 2 -bi| 2 ), 



;Mf? + g? 



where, 



/« = ^Erz( 7lb ( 
■ 9i = ^E 



vki 



k 



(9) 
(10) 

(11) 
(12) 



kcr 



It is important to recall that when D = 0, i.e., in the 
absence of the SOC, A_ k = — A k , the SU(2) symme- 
try is restored. Then there always exists a pair of zero 
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modes of w(k) with k = ±k*, condensed at zero temper- 
ature. However, for non-zero SOC, D / 0, we find that 
v4_k 7^ — in general and w(k) has only a single zero 
mode k* in the first Brillouin zone, so the expression for 
x— or y— component of the spin-spin correlation is dif- 
ferent from that of the z-component. According to Eq. 
(10), the spin-spin correlation between the z-components 
vanishes at zero temperature when the distance between 
two spins is sufficiently large. Moreover, there exits 
one independent nonvanishing off-diagonal component, 
{S v Sf) = -{S$S?) = \lm(!l + gf). All these features 
are in contrast with the cases studied previousl y 16 ' 18 ' 19 . 
Now we discuss the numerical results for the infinite 
system. By converting the sum in the mean-field equa- 
tions (O and ((SJ) into integrals and denoting the contri- 
bution from the Bose condensate as bo, the mean-field 
equations for numerical performance are 



0.75 



0.70 



0.65 
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2 3 
J'/J 



2/3 



d 2 k A 

(2tt) 2 2^ 



= 1 - h, 



d 2 k A k e- ik ' 
(2tt) 2 2^7 



(13) 
(14) 



where B = jA^e- lk * ^b if b > 0, and B Q = if 
bo < 0. To have a gapless spectrum, A is always fixed 
at the largest value of |j4k|. If the solution is associated 
with a negative bo, we can always fulfill the constraint (jjj) 
by tuning up A slightly. In this case, the energy spectrum 
is gapped and the system is in the spin liquid phase. If 
there is a solution with a positive bo, then the system is 
in the condensation phase. The magnetic order can be 
obtained from the spin-spin correlation functions. They 
are (SgSf) = (S%S?) = &2 CO s(2k*-i) and (SgSf) = 
-(S8S?) = &§sin(2k*-i). By denoting k* = {kl,k* 2 ), 
we have the ordering wave vectors, along the directions 
ofai, SL2 anda3, being 2k*, 2k 2 and — 2(k*+k 2 ), modulo- 
divided by 2-7T, respectively. 

The numerical solutions for k* show that k* = k 2 = k*. 
The curves of k* as a function of a = J'/J arc plotted in 
FigEJ When D is very small, the ordering wave vector 
along the chain tends to n near the decoupled-chain limit 
(a <C 1), indicating an antiferromagnetic order. While 
when a»l, the ordering wave vectors along the direc- 
tions of ai, &2 tend to ir, ir, respectively, reproducing 
the checkerboard or Neel ordering for the unfrustrated 
antiferromagnet in a square lattice. Between the two 
limits the spiral order develops and the order parame- 
ter oscillates with the distance. With the increase of D, 
the curves of k* change dramatically. Especially, when 
|7T is independent of a. For generic D, the 



D 



±7T, k* 
6 ' 



limiting values of A:* with small and large a are j7r ■ 
and \k + D respectively. It is also interesting to note 
that at the isotropic point (a = 1), k* always equals to 
|7r regardless of D. 

The square-root of the amplitude of the spin-spin cor- 
relation function, characterizing the strength of the mag- 
netic order, is exactly the Bose condensation density bo- 
In the case of the antiferromagnetic order, bo is equal 



FIG. 2: The condensation momentum k* = (k* , k*) as a func- 
tion of J'/J for D = 0.01-7T, 0.1-7T, i-7r and 0.2-k respectively. 




FIG. 3: The Bose condensation density bo as a function of 
J'/J for D — 0.01-7T, 0.17T and ^7r respectively. 



to the sublatticc magnetization. It is also a function of 
D with period 7r/3 since the SOC can be related to the 
phase factor in the triangular lattice. As shown in Fig 
[3l bo is enhanced monotonically when D increases up to 
7r/6. For each D, b takes a maximum value at a = 1. 

For each fixed D there is a critical value a c = J' c /J 
separating the spin liquid phase from the magnetic or- 
dered phase. For a < a c , bo — and all the spin-spin 
correlation functions decay exponentially to zero as the 
distance i — > oo. For a > a c , the bosons condense on the 
state with momentum k* . Fig. @] plots the critical line 
of the quantum phase transition. As a function of D, 
the critical line oscillates with period ir/3 as expected. 
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FIG. 4: The mean-field phase boundary between the magnetic 
ordered state and the spin liquid state. 



In realistic materials, the strength of the SOC is usually 
small compared to the hopping integral and the large D 
in the phase diagram is unphysical. However, large value 
of D is achievable in the optical flux lattice^. We find 
that the nonzero D (within the half-period 7r/6) pushes 
the boundary to the region with a smaller a, thus it ef- 
fectively enhances the magnetic ordering. This feature is 
compatible with the D-dependence of bo. When D = -|, 
a c reaches the minimum value 0.095. This result implies 
that with relatively large SOC the spiral magnetic order 
can emerge in the triangular lattice which has a tendency 



of reducing dimensionality as featured by small a. 

Finally, we remark that our mean-field treatment em- 
phasizes the magnetic ordering but is not adequate to 
capture the disordered state which may appear in the 
region with the moderate spatial anisotropy. Focusing 
on the spiral ordered phase with small anisotropy, it is 
possible that the quantum fluctuations could blur the 
difference between the solutions with two condensation 
momenta and a single momentum, giving rise to a finite 
spin-spin correlation between z-components. While this 
issue can be clarified in the future by taking into account 
the quantum fluctuations above the mean-field solution, 
it is robust that the SOC induces a nonzero spin corre- 
lation between the x- and y-components, suppresses the 
spin correlation between the z-components and favors the 
establishment of the spiral order. 

In summary, a large-U Hubbard model with the SOC 
on an spatial anisotropic lattice at half-filling is studied 
using the Schwinger- boson method. With the partici- 
pation of the SOC, the condensation momentum of the 
Schwinger bosons have only a single zero mode which 
in turn leads to a finite spin-spin correlation between x- 
and y-components and a vanishing spin-spin correlation 
between ^-components at large distance. The SOC also 
shifts the phase boundary between the magnetic ordered 
state and the spin liquid state to the larger anisotropy 
side, and stabilizes the spiral magnetic order. Our re- 
sults provide an alternative understanding of the spiral 
order observed in some materials like CsiCuCl^ where 
the triangular lattice has a relatively small a. 
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